This application relates to image processing in digital camera, video, and other electronic digital image acquisition and or display devices, and particularly to techniques of improving the apparent dynamic range of such images.
Most digital images encode only a small fraction of the intensities a human observer can see in a real scene. Detail visible to the human eye is lost in dim and bright portions of the image due to the limited dynamic range of the image sensor and/or display device. For example, current image sensors provide contrast ratios in the range of 200:1 (a dynamic range of about 46 dB). Even if the raw dynamic range of the image-sensing device were to improve by a factor of 5, to 60 dB, further improvements would still be required in order to approach the performance of the human visual cortex, having a dynamic range approaching 80 dB.
Various methods have been suggested over the past decades for input-output intensity mapping of digital image data, in order to enhance the perception of image details at the extreme ends of a system's dynamic range. Methods fall into two broad classes, iterative solutions and non-linear filtering. Iterative solutions gradually, and repeatedly, modify an initial image towards infinite exposure time by employing a discretized partial differential equation (PDE), such as used to emulate heat transfer (see, for example, Choudhury and Tumblin, “The Trilateral Filter for High Contrast Images and Meshes”, Eurographics Symposium on Rendering 2003, pp. 1-11, 2003). These methods combine smoothing and edge sharpening into a single iterative process. Anisotropic diffusion (see, for example, Perona and Malik, “Scale space and edge detection using anisotropic diffusion”, IEEE Transaction Pattern Analysis and Machine Intelligence, vol. 12(7), pp. 629-639, 1990) and gradient (see, for example, Fattal et at., “Gradient domain high dynamic range compensation”, ACM Transactions on Graphics, special issue on Proc. Of ACM SIG-GRAPH 2002, San Antonio, Tex., vol, 21(3), pp. 257-266, 2002) approaches are among these methods. Nonlinear filter methods compute each output pixel separately as a position-dependent function of input pixels in a local neighborhood. Non-Linear filters obtain good-quality edge preserving smoothing in a single pass. They can produce PDE like results without a time-consuming iterative solution or possible risks of instability (e.g., Choudhury and Tumblin, 2003, cited above). This class starts with Edwin H. Land's classic Retinex work and continued by others, which led to Bilateral-filtering (see, for example, Tomasi and Manduchi, “Bilateral filtering of gray and colored images”, Proc. IEEE Intl. Conference on Computer Vision, pp. 836-846, 1998, or Duran and Dorsey, “Fast bilateral filtering for the display of high-dynamic range images”, ACM Transactions on Graphics, special issue on Proc. Of ACM SIG-GRAPH 2002, San Antonio, Tex., vol. 21(3), pp. 249-256, 2000).
These various prior art methods tend to have a number of shortcoming when it come to implementation in digital cameras and video, such as the amount of processing power needed for setting or predetermining the coefficients which weight scaled filtered images. Consequently, given the difference in dynamic range between digital image systems and the human visual system, there is substantial room for improvements in digital imaging systems.